{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "Use this notebook to finetune a ConvNeXt-tiny model on CIFAR 10 dataset. The [official ConvNeXt repository](https://github.com/facebookresearch/ConvNeXt) is instrumented with [Weights and Biases](https://wandb.ai/site). You can now easily log your train/test metrics and version control your model checkpoints to Weigths and Biases"
      ],
      "metadata": {
        "id": "LniKjqdogsrH"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# ⚽️ Installation and Setup\n",
        "\n",
        "The following installation instruction is based on [INSTALL.md](https://github.com/facebookresearch/ConvNeXt/blob/main/INSTALL.md) provided by the official ConvNeXt repository."
      ],
      "metadata": {
        "id": "1JS4ffXFRnRr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!conda create -n convnext python=3.8 -y\n",
        "!conda activate convnext"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eWwPDjMZeGYK",
        "outputId": "49decc4b-e39b-42d0-bf1e-f6e2a9c09432"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "/bin/bash: line 1: conda: command not found\n",
            "/bin/bash: line 1: conda: command not found\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install torch==2.0.1+cu118 torchvision==0.15.2+cu118 -f https://download.pytorch.org/whl/torch_stable.html\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jCvgPoh2eWmL",
        "outputId": "740e42e2-afba-4e54-f243-3e1e67639c25"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in links: https://download.pytorch.org/whl/torch_stable.html\n",
            "Collecting torch==2.0.1+cu118\n",
            "  Downloading https://download.pytorch.org/whl/cu118/torch-2.0.1%2Bcu118-cp311-cp311-linux_x86_64.whl (2267.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.3/2.3 GB\u001b[0m \u001b[31m435.2 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting torchvision==0.15.2+cu118\n",
            "  Downloading https://download.pytorch.org/whl/cu118/torchvision-0.15.2%2Bcu118-cp311-cp311-linux_x86_64.whl (6.1 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.1/6.1 MB\u001b[0m \u001b[31m99.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.11/dist-packages (from torch==2.0.1+cu118) (3.18.0)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.11/dist-packages (from torch==2.0.1+cu118) (4.13.2)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.11/dist-packages (from torch==2.0.1+cu118) (1.13.1)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.11/dist-packages (from torch==2.0.1+cu118) (3.4.2)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.11/dist-packages (from torch==2.0.1+cu118) (3.1.6)\n",
            "Collecting triton==2.0.0 (from torch==2.0.1+cu118)\n",
            "  Downloading triton-2.0.0-1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.0 kB)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.11/dist-packages (from torchvision==0.15.2+cu118) (2.0.2)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.11/dist-packages (from torchvision==0.15.2+cu118) (2.32.3)\n",
            "Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.11/dist-packages (from torchvision==0.15.2+cu118) (11.2.1)\n",
            "Requirement already satisfied: cmake in /usr/local/lib/python3.11/dist-packages (from triton==2.0.0->torch==2.0.1+cu118) (3.31.6)\n",
            "Collecting lit (from triton==2.0.0->torch==2.0.1+cu118)\n",
            "  Downloading lit-18.1.8-py3-none-any.whl.metadata (2.5 kB)\n",
            "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.11/dist-packages (from jinja2->torch==2.0.1+cu118) (3.0.2)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/dist-packages (from requests->torchvision==0.15.2+cu118) (3.4.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.11/dist-packages (from requests->torchvision==0.15.2+cu118) (3.10)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/dist-packages (from requests->torchvision==0.15.2+cu118) (2.4.0)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.11/dist-packages (from requests->torchvision==0.15.2+cu118) (2025.4.26)\n",
            "Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.11/dist-packages (from sympy->torch==2.0.1+cu118) (1.3.0)\n",
            "Downloading triton-2.0.0-1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (63.3 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m63.3/63.3 MB\u001b[0m \u001b[31m12.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading lit-18.1.8-py3-none-any.whl (96 kB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m96.4/96.4 kB\u001b[0m \u001b[31m9.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hInstalling collected packages: lit, triton, torch, torchvision\n",
            "  Attempting uninstall: triton\n",
            "    Found existing installation: triton 3.2.0\n",
            "    Uninstalling triton-3.2.0:\n",
            "      Successfully uninstalled triton-3.2.0\n",
            "  Attempting uninstall: torch\n",
            "    Found existing installation: torch 2.6.0+cu124\n",
            "    Uninstalling torch-2.6.0+cu124:\n",
            "      Successfully uninstalled torch-2.6.0+cu124\n",
            "  Attempting uninstall: torchvision\n",
            "    Found existing installation: torchvision 0.21.0+cu124\n",
            "    Uninstalling torchvision-0.21.0+cu124:\n",
            "      Successfully uninstalled torchvision-0.21.0+cu124\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "torchaudio 2.6.0+cu124 requires torch==2.6.0, but you have torch 2.0.1+cu118 which is incompatible.\u001b[0m\u001b[31m\n",
            "\u001b[0mSuccessfully installed lit-18.1.8 torch-2.0.1+cu118 torchvision-0.15.2+cu118 triton-2.0.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install -qq torch==1.8.0+cu111 torchvision==0.9.0+cu111 -f https://download.pytorch.org/whl/torch_stable.html\n",
        "!pip install -qq wandb timm==0.3.2 six tensorboardX"
      ],
      "metadata": {
        "id": "5YbEGpKrDKC5",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e40fafc4-2f26-4402-e7f6-7dccb341a5aa"
      },
      "execution_count": 2,
      "outputs": [
        {
          "metadata": {
            "tags": null
          },
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[31mERROR: Could not find a version that satisfies the requirement torch==1.8.0+cu111 (from versions: 1.13.0, 1.13.0+cpu, 1.13.0+cu116, 1.13.0+cu117, 1.13.0+cu117.with.pypi.cudnn, 1.13.1, 1.13.1+cpu, 1.13.1+cu116, 1.13.1+cu117, 1.13.1+cu117.with.pypi.cudnn, 2.0.0, 2.0.0+cpu, 2.0.0+cpu.cxx11.abi, 2.0.0+cu117, 2.0.0+cu117.with.pypi.cudnn, 2.0.0+cu118, 2.0.1, 2.0.1+cpu, 2.0.1+cpu.cxx11.abi, 2.0.1+cu117, 2.0.1+cu117.with.pypi.cudnn, 2.0.1+cu118, 2.0.1+rocm5.3, 2.0.1+rocm5.4.2, 2.1.0, 2.1.0+cpu, 2.1.0+cpu.cxx11.abi, 2.1.0+cu118, 2.1.0+cu121, 2.1.0+cu121.with.pypi.cudnn, 2.1.0+rocm5.5, 2.1.0+rocm5.6, 2.1.1, 2.1.1+cpu, 2.1.1+cpu.cxx11.abi, 2.1.1+cu118, 2.1.1+cu121, 2.1.1+cu121.with.pypi.cudnn, 2.1.1+rocm5.5, 2.1.1+rocm5.6, 2.1.2, 2.1.2+cpu, 2.1.2+cpu.cxx11.abi, 2.1.2+cu118, 2.1.2+cu121, 2.1.2+cu121.with.pypi.cudnn, 2.1.2+rocm5.5, 2.1.2+rocm5.6, 2.2.0, 2.2.0+cpu, 2.2.0+cpu.cxx11.abi, 2.2.0+cu118, 2.2.0+cu121, 2.2.0+rocm5.6, 2.2.0+rocm5.7, 2.2.1, 2.2.1+cpu, 2.2.1+cpu.cxx11.abi, 2.2.1+cu118, 2.2.1+cu121, 2.2.1+rocm5.6, 2.2.1+rocm5.7, 2.2.2, 2.2.2+cpu, 2.2.2+cpu.cxx11.abi, 2.2.2+cu118, 2.2.2+cu121, 2.2.2+rocm5.6, 2.2.2+rocm5.7, 2.3.0, 2.3.0+cpu, 2.3.0+cpu.cxx11.abi, 2.3.0+cu118, 2.3.0+cu121, 2.3.0+rocm5.7, 2.3.0+rocm6.0, 2.3.1, 2.3.1+cpu, 2.3.1+cpu.cxx11.abi, 2.3.1+cu118, 2.3.1+cu121, 2.3.1+rocm5.7, 2.3.1+rocm6.0, 2.4.0, 2.4.1, 2.5.0, 2.5.1, 2.6.0, 2.7.0)\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mERROR: No matching distribution found for torch==1.8.0+cu111\u001b[0m\u001b[31m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m244.2/244.2 kB\u001b[0m \u001b[31m9.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m101.7/101.7 kB\u001b[0m \u001b[31m10.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m363.4/363.4 MB\u001b[0m \u001b[31m4.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m13.8/13.8 MB\u001b[0m \u001b[31m73.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m24.6/24.6 MB\u001b[0m \u001b[31m69.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m883.7/883.7 kB\u001b[0m \u001b[31m51.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m664.8/664.8 MB\u001b[0m \u001b[31m2.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m211.5/211.5 MB\u001b[0m \u001b[31m7.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m56.3/56.3 MB\u001b[0m \u001b[31m12.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m127.9/127.9 MB\u001b[0m \u001b[31m8.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m207.5/207.5 MB\u001b[0m \u001b[31m6.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m21.1/21.1 MB\u001b[0m \u001b[31m71.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Download the official ConvNeXt respository."
      ],
      "metadata": {
        "id": "kDXQ-EpX9fsB"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!git clone https://github.com/facebookresearch/ConvNeXt"
      ],
      "metadata": {
        "id": "zmmHO1Cp4E90",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4a089e30-daac-46f8-a52c-95c766babe2c"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "fatal: destination path 'ConvNeXt' already exists and is not an empty directory.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 🏀 Download the Dataset\n",
        "\n",
        "We will be finetuning on CIFAR-10 dataset. To use any custom dataset (CIFAR-10 here) the format of the dataset should be as shown below:\n",
        "\n",
        "```\n",
        "/path/to/dataset/\n",
        "  train/\n",
        "    class1/\n",
        "      img1.jpeg\n",
        "    class2/\n",
        "      img2.jpeg\n",
        "  val/\n",
        "    class1/\n",
        "      img3.jpeg\n",
        "    class2/\n",
        "      img4.jpeg\n",
        "```\n",
        "\n",
        "I have used this [repository](https://github.com/YoongiKim/CIFAR-10-images) that has the CIFAR-10 images in the required format."
      ],
      "metadata": {
        "id": "yoVwkQ0v80KW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!git clone https://github.com/YoongiKim/CIFAR-10-images"
      ],
      "metadata": {
        "id": "8xcQ6QV41k8S",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d34138e0-0509-48a4-b611-a7145506804b"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Cloning into 'CIFAR-10-images'...\n",
            "remote: Enumerating objects: 60027, done.\u001b[K\n",
            "remote: Total 60027 (delta 0), reused 0 (delta 0), pack-reused 60027 (from 1)\u001b[K\n",
            "Receiving objects: 100% (60027/60027), 19.94 MiB | 41.76 MiB/s, done.\n",
            "Resolving deltas: 100% (59990/59990), done.\n",
            "Updating files: 100% (60001/60001), done.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#CIFAR100\n",
        "\n",
        "from torchvision import datasets, transforms\n",
        "\n",
        "transform = transforms.Compose([\n",
        "    transforms.ToTensor(),\n",
        "    transforms.Normalize((0.5071, 0.4865, 0.4409), (0.2673, 0.2564, 0.2762))\n",
        "])\n",
        "\n",
        "train_dataset = datasets.CIFAR100(root='./data', train=True, download=True, transform=transform)\n",
        "test_dataset = datasets.CIFAR100(root='./data', train=False, download=True, transform=transform)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qmTq7PGkGdaQ",
        "outputId": "d464eef8-a500-4018-ec7b-2706f864791f"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\n",
            "A module that was compiled using NumPy 1.x cannot be run in\n",
            "NumPy 2.0.2 as it may crash. To support both 1.x and 2.x\n",
            "versions of NumPy, modules must be compiled with NumPy 2.0.\n",
            "Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.\n",
            "\n",
            "If you are a user of the module, the easiest solution will be to\n",
            "downgrade to 'numpy<2' or try to upgrade the affected module.\n",
            "We expect that some modules will need time to support NumPy 2.\n",
            "\n",
            "Traceback (most recent call last):  File \"<frozen runpy>\", line 198, in _run_module_as_main\n",
            "  File \"<frozen runpy>\", line 88, in _run_code\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/colab_kernel_launcher.py\", line 37, in <module>\n",
            "    ColabKernelApp.launch_instance()\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/traitlets/config/application.py\", line 992, in launch_instance\n",
            "    app.start()\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/ipykernel/kernelapp.py\", line 712, in start\n",
            "    self.io_loop.start()\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/tornado/platform/asyncio.py\", line 205, in start\n",
            "    self.asyncio_loop.run_forever()\n",
            "  File \"/usr/lib/python3.11/asyncio/base_events.py\", line 608, in run_forever\n",
            "    self._run_once()\n",
            "  File \"/usr/lib/python3.11/asyncio/base_events.py\", line 1936, in _run_once\n",
            "    handle._run()\n",
            "  File \"/usr/lib/python3.11/asyncio/events.py\", line 84, in _run\n",
            "    self._context.run(self._callback, *self._args)\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/ipykernel/kernelbase.py\", line 510, in dispatch_queue\n",
            "    await self.process_one()\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/ipykernel/kernelbase.py\", line 499, in process_one\n",
            "    await dispatch(*args)\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/ipykernel/kernelbase.py\", line 406, in dispatch_shell\n",
            "    await result\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/ipykernel/kernelbase.py\", line 730, in execute_request\n",
            "    reply_content = await reply_content\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/ipykernel/ipkernel.py\", line 383, in do_execute\n",
            "    res = shell.run_cell(\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/ipykernel/zmqshell.py\", line 528, in run_cell\n",
            "    return super().run_cell(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/IPython/core/interactiveshell.py\", line 2975, in run_cell\n",
            "    result = self._run_cell(\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/IPython/core/interactiveshell.py\", line 3030, in _run_cell\n",
            "    return runner(coro)\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/IPython/core/async_helpers.py\", line 78, in _pseudo_sync_runner\n",
            "    coro.send(None)\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/IPython/core/interactiveshell.py\", line 3257, in run_cell_async\n",
            "    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/IPython/core/interactiveshell.py\", line 3473, in run_ast_nodes\n",
            "    if (await self.run_code(code, result,  async_=asy)):\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/IPython/core/interactiveshell.py\", line 3553, in run_code\n",
            "    exec(code_obj, self.user_global_ns, self.user_ns)\n",
            "  File \"<ipython-input-7-21c2874a0ea2>\", line 3, in <cell line: 0>\n",
            "    from torchvision import datasets, transforms\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/torchvision/__init__.py\", line 6, in <module>\n",
            "    from torchvision import datasets, io, models, ops, transforms, utils\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/torchvision/models/__init__.py\", line 17, in <module>\n",
            "    from . import detection, optical_flow, quantization, segmentation, video\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/torchvision/models/detection/__init__.py\", line 1, in <module>\n",
            "    from .faster_rcnn import *\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/torchvision/models/detection/faster_rcnn.py\", line 16, in <module>\n",
            "    from .anchor_utils import AnchorGenerator\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/torchvision/models/detection/anchor_utils.py\", line 10, in <module>\n",
            "    class AnchorGenerator(nn.Module):\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/torchvision/models/detection/anchor_utils.py\", line 63, in AnchorGenerator\n",
            "    device: torch.device = torch.device(\"cpu\"),\n",
            "/usr/local/lib/python3.11/dist-packages/torchvision/models/detection/anchor_utils.py:63: UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:84.)\n",
            "  device: torch.device = torch.device(\"cpu\"),\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz to ./data/cifar-100-python.tar.gz\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 169001437/169001437 [00:04<00:00, 35326356.23it/s]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracting ./data/cifar-100-python.tar.gz to ./data\n",
            "Files already downloaded and verified\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 🏈 Download Pretrained Weights\n",
        "\n",
        "We will be finetuning the ConvNeXt Tiny model pretrained on ImageNet 1K dataset."
      ],
      "metadata": {
        "id": "J6qUVfL29tH1"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%cd ConvNeXt/\n",
        "!wget https://dl.fbaipublicfiles.com/convnext/convnext_tiny_1k_224_ema.pth"
      ],
      "metadata": {
        "id": "TYPDl5bT8LZ5",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "80c8cd2c-314a-457d-82f2-79ccf86cafbf"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "/content/ConvNeXt\n",
            "--2025-05-29 23:59:15--  https://dl.fbaipublicfiles.com/convnext/convnext_tiny_1k_224_ema.pth\n",
            "Resolving dl.fbaipublicfiles.com (dl.fbaipublicfiles.com)... 3.163.189.108, 3.163.189.96, 3.163.189.51, ...\n",
            "Connecting to dl.fbaipublicfiles.com (dl.fbaipublicfiles.com)|3.163.189.108|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 114414741 (109M) [binary/octet-stream]\n",
            "Saving to: ‘convnext_tiny_1k_224_ema.pth’\n",
            "\n",
            "convnext_tiny_1k_22 100%[===================>] 109.11M   171MB/s    in 0.6s    \n",
            "\n",
            "2025-05-29 23:59:16 (171 MB/s) - ‘convnext_tiny_1k_224_ema.pth’ saved [114414741/114414741]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 🎾 Train with Weights and Biases\n",
        "\n",
        "If you want to log the train and evaluation metrics using Weights and Biases pass `--enable_wandb true`.\n",
        "\n",
        "You can also save the finetuned checkpoints as version controlled W&B [Artifacts](https://docs.wandb.ai/guides/artifacts) if you pass `--wandb_ckpt true`.\n",
        "\n"
      ],
      "metadata": {
        "id": "pSPgPCjp-Lro"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install numpy==1.24.4\n",
        "!pip install timm==0.9.12\n",
        "!pip install torch==2.2.2 torchvision==0.17.2\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "Kdqca5QgfayI",
        "outputId": "576eabde-28c1-41b1-8100-97d65fe4649c"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting numpy==1.24.4\n",
            "  Downloading numpy-1.24.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.6 kB)\n",
            "Downloading numpy-1.24.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m17.3/17.3 MB\u001b[0m \u001b[31m102.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hInstalling collected packages: numpy\n",
            "  Attempting uninstall: numpy\n",
            "    Found existing installation: numpy 2.0.2\n",
            "    Uninstalling numpy-2.0.2:\n",
            "      Successfully uninstalled numpy-2.0.2\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "tensorflow 2.18.0 requires numpy<2.1.0,>=1.26.0, but you have numpy 1.24.4 which is incompatible.\n",
            "thinc 8.3.6 requires numpy<3.0.0,>=2.0.0, but you have numpy 1.24.4 which is incompatible.\n",
            "treescope 0.1.9 requires numpy>=1.25.2, but you have numpy 1.24.4 which is incompatible.\n",
            "jax 0.5.2 requires numpy>=1.25, but you have numpy 1.24.4 which is incompatible.\n",
            "pymc 5.22.0 requires numpy>=1.25.0, but you have numpy 1.24.4 which is incompatible.\n",
            "blosc2 3.3.3 requires numpy>=1.26, but you have numpy 1.24.4 which is incompatible.\n",
            "jaxlib 0.5.1 requires numpy>=1.25, but you have numpy 1.24.4 which is incompatible.\u001b[0m\u001b[31m\n",
            "\u001b[0mSuccessfully installed numpy-1.24.4\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.colab-display-data+json": {
              "pip_warning": {
                "packages": [
                  "numpy"
                ]
              },
              "id": "2084c2841aee49a4b4507009ae218389"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting timm==0.9.12\n",
            "  Downloading timm-0.9.12-py3-none-any.whl.metadata (60 kB)\n",
            "\u001b[?25l     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/60.6 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m60.6/60.6 kB\u001b[0m \u001b[31m3.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: torch>=1.7 in /usr/local/lib/python3.11/dist-packages (from timm==0.9.12) (2.0.1+cu118)\n",
            "Requirement already satisfied: torchvision in /usr/local/lib/python3.11/dist-packages (from timm==0.9.12) (0.15.2+cu118)\n",
            "Requirement already satisfied: pyyaml in /usr/local/lib/python3.11/dist-packages (from timm==0.9.12) (6.0.2)\n",
            "Requirement already satisfied: huggingface-hub in /usr/local/lib/python3.11/dist-packages (from timm==0.9.12) (0.31.4)\n",
            "Requirement already satisfied: safetensors in /usr/local/lib/python3.11/dist-packages (from timm==0.9.12) (0.5.3)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.11/dist-packages (from torch>=1.7->timm==0.9.12) (3.18.0)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.11/dist-packages (from torch>=1.7->timm==0.9.12) (4.13.2)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.11/dist-packages (from torch>=1.7->timm==0.9.12) (1.13.1)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.11/dist-packages (from torch>=1.7->timm==0.9.12) (3.4.2)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.11/dist-packages (from torch>=1.7->timm==0.9.12) (3.1.6)\n",
            "Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.11/dist-packages (from torch>=1.7->timm==0.9.12) (2.0.0)\n",
            "Requirement already satisfied: cmake in /usr/local/lib/python3.11/dist-packages (from triton==2.0.0->torch>=1.7->timm==0.9.12) (3.31.6)\n",
            "Requirement already satisfied: lit in /usr/local/lib/python3.11/dist-packages (from triton==2.0.0->torch>=1.7->timm==0.9.12) (18.1.8)\n",
            "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.11/dist-packages (from huggingface-hub->timm==0.9.12) (2025.3.2)\n",
            "Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.11/dist-packages (from huggingface-hub->timm==0.9.12) (24.2)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.11/dist-packages (from huggingface-hub->timm==0.9.12) (2.32.3)\n",
            "Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.11/dist-packages (from huggingface-hub->timm==0.9.12) (4.67.1)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.11/dist-packages (from torchvision->timm==0.9.12) (1.24.4)\n",
            "Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.11/dist-packages (from torchvision->timm==0.9.12) (11.2.1)\n",
            "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.11/dist-packages (from jinja2->torch>=1.7->timm==0.9.12) (3.0.2)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/dist-packages (from requests->huggingface-hub->timm==0.9.12) (3.4.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.11/dist-packages (from requests->huggingface-hub->timm==0.9.12) (3.10)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/dist-packages (from requests->huggingface-hub->timm==0.9.12) (2.4.0)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.11/dist-packages (from requests->huggingface-hub->timm==0.9.12) (2025.4.26)\n",
            "Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.11/dist-packages (from sympy->torch>=1.7->timm==0.9.12) (1.3.0)\n",
            "Downloading timm-0.9.12-py3-none-any.whl (2.2 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.2/2.2 MB\u001b[0m \u001b[31m39.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hInstalling collected packages: timm\n",
            "  Attempting uninstall: timm\n",
            "    Found existing installation: timm 0.3.2\n",
            "    Uninstalling timm-0.3.2:\n",
            "      Successfully uninstalled timm-0.3.2\n",
            "Successfully installed timm-0.9.12\n",
            "Collecting torch==2.2.2\n",
            "  Downloading torch-2.2.2-cp311-cp311-manylinux1_x86_64.whl.metadata (25 kB)\n",
            "Collecting torchvision==0.17.2\n",
            "  Downloading torchvision-0.17.2-cp311-cp311-manylinux1_x86_64.whl.metadata (6.6 kB)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.11/dist-packages (from torch==2.2.2) (3.18.0)\n",
            "Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.11/dist-packages (from torch==2.2.2) (4.13.2)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.11/dist-packages (from torch==2.2.2) (1.13.1)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.11/dist-packages (from torch==2.2.2) (3.4.2)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.11/dist-packages (from torch==2.2.2) (3.1.6)\n",
            "Requirement already satisfied: fsspec in /usr/local/lib/python3.11/dist-packages (from torch==2.2.2) (2025.3.2)\n",
            "Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch==2.2.2)\n",
            "  Downloading nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n",
            "Collecting nvidia-cuda-runtime-cu12==12.1.105 (from torch==2.2.2)\n",
            "  Downloading nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n",
            "Collecting nvidia-cuda-cupti-cu12==12.1.105 (from torch==2.2.2)\n",
            "  Downloading nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)\n",
            "Collecting nvidia-cudnn-cu12==8.9.2.26 (from torch==2.2.2)\n",
            "  Downloading nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)\n",
            "Collecting nvidia-cublas-cu12==12.1.3.1 (from torch==2.2.2)\n",
            "  Downloading nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n",
            "Collecting nvidia-cufft-cu12==11.0.2.54 (from torch==2.2.2)\n",
            "  Downloading nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n",
            "Collecting nvidia-curand-cu12==10.3.2.106 (from torch==2.2.2)\n",
            "  Downloading nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl.metadata (1.5 kB)\n",
            "Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch==2.2.2)\n",
            "  Downloading nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)\n",
            "Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch==2.2.2)\n",
            "  Downloading nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl.metadata (1.6 kB)\n",
            "Collecting nvidia-nccl-cu12==2.19.3 (from torch==2.2.2)\n",
            "  Downloading nvidia_nccl_cu12-2.19.3-py3-none-manylinux1_x86_64.whl.metadata (1.8 kB)\n",
            "Collecting nvidia-nvtx-cu12==12.1.105 (from torch==2.2.2)\n",
            "  Downloading nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl.metadata (1.7 kB)\n",
            "Collecting triton==2.2.0 (from torch==2.2.2)\n",
            "  Downloading triton-2.2.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.4 kB)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.11/dist-packages (from torchvision==0.17.2) (1.24.4)\n",
            "Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.11/dist-packages (from torchvision==0.17.2) (11.2.1)\n",
            "Requirement already satisfied: nvidia-nvjitlink-cu12 in /usr/local/lib/python3.11/dist-packages (from nvidia-cusolver-cu12==11.4.5.107->torch==2.2.2) (12.4.127)\n",
            "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.11/dist-packages (from jinja2->torch==2.2.2) (3.0.2)\n",
            "Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.11/dist-packages (from sympy->torch==2.2.2) (1.3.0)\n",
            "Downloading torch-2.2.2-cp311-cp311-manylinux1_x86_64.whl (755.6 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m755.6/755.6 MB\u001b[0m \u001b[31m1.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading torchvision-0.17.2-cp311-cp311-manylinux1_x86_64.whl (6.9 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.9/6.9 MB\u001b[0m \u001b[31m93.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m410.6/410.6 MB\u001b[0m \u001b[31m3.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (14.1 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m14.1/14.1 MB\u001b[0m \u001b[31m54.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (23.7 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m23.7/23.7 MB\u001b[0m \u001b[31m34.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m823.6/823.6 kB\u001b[0m \u001b[31m43.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl (731.7 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m731.7/731.7 MB\u001b[0m \u001b[31m1.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl (121.6 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m121.6/121.6 MB\u001b[0m \u001b[31m9.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl (56.5 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m56.5/56.5 MB\u001b[0m \u001b[31m12.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m124.2/124.2 MB\u001b[0m \u001b[31m9.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m196.0/196.0 MB\u001b[0m \u001b[31m5.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading nvidia_nccl_cu12-2.19.3-py3-none-manylinux1_x86_64.whl (166.0 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m166.0/166.0 MB\u001b[0m \u001b[31m7.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (99 kB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m99.1/99.1 kB\u001b[0m \u001b[31m10.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hDownloading triton-2.2.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (167.9 MB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m167.9/167.9 MB\u001b[0m \u001b[31m7.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hInstalling collected packages: triton, nvidia-nvtx-cu12, nvidia-nccl-cu12, nvidia-cusparse-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, nvidia-cusolver-cu12, nvidia-cudnn-cu12, torch, torchvision\n",
            "  Attempting uninstall: triton\n",
            "    Found existing installation: triton 2.0.0\n",
            "    Uninstalling triton-2.0.0:\n",
            "      Successfully uninstalled triton-2.0.0\n",
            "  Attempting uninstall: nvidia-nvtx-cu12\n",
            "    Found existing installation: nvidia-nvtx-cu12 12.4.127\n",
            "    Uninstalling nvidia-nvtx-cu12-12.4.127:\n",
            "      Successfully uninstalled nvidia-nvtx-cu12-12.4.127\n",
            "  Attempting uninstall: nvidia-nccl-cu12\n",
            "    Found existing installation: nvidia-nccl-cu12 2.21.5\n",
            "    Uninstalling nvidia-nccl-cu12-2.21.5:\n",
            "      Successfully uninstalled nvidia-nccl-cu12-2.21.5\n",
            "  Attempting uninstall: nvidia-cusparse-cu12\n",
            "    Found existing installation: nvidia-cusparse-cu12 12.3.1.170\n",
            "    Uninstalling nvidia-cusparse-cu12-12.3.1.170:\n",
            "      Successfully uninstalled nvidia-cusparse-cu12-12.3.1.170\n",
            "  Attempting uninstall: nvidia-curand-cu12\n",
            "    Found existing installation: nvidia-curand-cu12 10.3.5.147\n",
            "    Uninstalling nvidia-curand-cu12-10.3.5.147:\n",
            "      Successfully uninstalled nvidia-curand-cu12-10.3.5.147\n",
            "  Attempting uninstall: nvidia-cufft-cu12\n",
            "    Found existing installation: nvidia-cufft-cu12 11.2.1.3\n",
            "    Uninstalling nvidia-cufft-cu12-11.2.1.3:\n",
            "      Successfully uninstalled nvidia-cufft-cu12-11.2.1.3\n",
            "  Attempting uninstall: nvidia-cuda-runtime-cu12\n",
            "    Found existing installation: nvidia-cuda-runtime-cu12 12.4.127\n",
            "    Uninstalling nvidia-cuda-runtime-cu12-12.4.127:\n",
            "      Successfully uninstalled nvidia-cuda-runtime-cu12-12.4.127\n",
            "  Attempting uninstall: nvidia-cuda-nvrtc-cu12\n",
            "    Found existing installation: nvidia-cuda-nvrtc-cu12 12.4.127\n",
            "    Uninstalling nvidia-cuda-nvrtc-cu12-12.4.127:\n",
            "      Successfully uninstalled nvidia-cuda-nvrtc-cu12-12.4.127\n",
            "  Attempting uninstall: nvidia-cuda-cupti-cu12\n",
            "    Found existing installation: nvidia-cuda-cupti-cu12 12.4.127\n",
            "    Uninstalling nvidia-cuda-cupti-cu12-12.4.127:\n",
            "      Successfully uninstalled nvidia-cuda-cupti-cu12-12.4.127\n",
            "  Attempting uninstall: nvidia-cublas-cu12\n",
            "    Found existing installation: nvidia-cublas-cu12 12.4.5.8\n",
            "    Uninstalling nvidia-cublas-cu12-12.4.5.8:\n",
            "      Successfully uninstalled nvidia-cublas-cu12-12.4.5.8\n",
            "  Attempting uninstall: nvidia-cusolver-cu12\n",
            "    Found existing installation: nvidia-cusolver-cu12 11.6.1.9\n",
            "    Uninstalling nvidia-cusolver-cu12-11.6.1.9:\n",
            "      Successfully uninstalled nvidia-cusolver-cu12-11.6.1.9\n",
            "  Attempting uninstall: nvidia-cudnn-cu12\n",
            "    Found existing installation: nvidia-cudnn-cu12 9.1.0.70\n",
            "    Uninstalling nvidia-cudnn-cu12-9.1.0.70:\n",
            "      Successfully uninstalled nvidia-cudnn-cu12-9.1.0.70\n",
            "  Attempting uninstall: torch\n",
            "    Found existing installation: torch 2.0.1+cu118\n",
            "    Uninstalling torch-2.0.1+cu118:\n",
            "      Successfully uninstalled torch-2.0.1+cu118\n",
            "  Attempting uninstall: torchvision\n",
            "    Found existing installation: torchvision 0.15.2+cu118\n",
            "    Uninstalling torchvision-0.15.2+cu118:\n",
            "      Successfully uninstalled torchvision-0.15.2+cu118\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "torchaudio 2.6.0+cu124 requires torch==2.6.0, but you have torch 2.2.2 which is incompatible.\u001b[0m\u001b[31m\n",
            "\u001b[0mSuccessfully installed nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nccl-cu12-2.19.3 nvidia-nvtx-cu12-12.1.105 torch-2.2.2 torchvision-0.17.2 triton-2.2.0\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.colab-display-data+json": {
              "pip_warning": {
                "packages": [
                  "torch",
                  "torchvision"
                ]
              },
              "id": "043d068a4ae241caa1d05ac083ce52db"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# 从 GitHub 克隆\n",
        "!git clone https://github.com/facebookresearch/ConvNeXt.git\n",
        "!cd ConvNeXt\n",
        "!python3 main.py\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6-NWdEY5gSPq",
        "outputId": "418339da-f18a-4dc4-f2d7-24d8cdc585cb"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "fatal: destination path 'ConvNeXt' already exists and is not an empty directory.\n",
            "python3: can't open file '/content/main.py': [Errno 2] No such file or directory\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!pip uninstall -y torch torchvision timm\n",
        "!pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 timm==0.4.12 -f https://download.pytorch.org/whl/torch_stable.html\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nYG95WwdhR6f",
        "outputId": "b4f170f1-9643-44da-ab32-bcb3c38761ce"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Found existing installation: torch 2.2.2\n",
            "Uninstalling torch-2.2.2:\n",
            "  Successfully uninstalled torch-2.2.2\n",
            "Found existing installation: torchvision 0.17.2\n",
            "Uninstalling torchvision-0.17.2:\n",
            "  Successfully uninstalled torchvision-0.17.2\n",
            "Found existing installation: timm 0.3.2\n",
            "Uninstalling timm-0.3.2:\n",
            "  Successfully uninstalled timm-0.3.2\n",
            "Looking in links: https://download.pytorch.org/whl/torch_stable.html\n",
            "\u001b[31mERROR: Could not find a version that satisfies the requirement torch==1.8.1+cu111 (from versions: 1.13.0, 1.13.0+cpu, 1.13.0+cu116, 1.13.0+cu117, 1.13.0+cu117.with.pypi.cudnn, 1.13.1, 1.13.1+cpu, 1.13.1+cu116, 1.13.1+cu117, 1.13.1+cu117.with.pypi.cudnn, 2.0.0, 2.0.0+cpu, 2.0.0+cpu.cxx11.abi, 2.0.0+cu117, 2.0.0+cu117.with.pypi.cudnn, 2.0.0+cu118, 2.0.1, 2.0.1+cpu, 2.0.1+cpu.cxx11.abi, 2.0.1+cu117, 2.0.1+cu117.with.pypi.cudnn, 2.0.1+cu118, 2.0.1+rocm5.3, 2.0.1+rocm5.4.2, 2.1.0, 2.1.0+cpu, 2.1.0+cpu.cxx11.abi, 2.1.0+cu118, 2.1.0+cu121, 2.1.0+cu121.with.pypi.cudnn, 2.1.0+rocm5.5, 2.1.0+rocm5.6, 2.1.1, 2.1.1+cpu, 2.1.1+cpu.cxx11.abi, 2.1.1+cu118, 2.1.1+cu121, 2.1.1+cu121.with.pypi.cudnn, 2.1.1+rocm5.5, 2.1.1+rocm5.6, 2.1.2, 2.1.2+cpu, 2.1.2+cpu.cxx11.abi, 2.1.2+cu118, 2.1.2+cu121, 2.1.2+cu121.with.pypi.cudnn, 2.1.2+rocm5.5, 2.1.2+rocm5.6, 2.2.0, 2.2.0+cpu, 2.2.0+cpu.cxx11.abi, 2.2.0+cu118, 2.2.0+cu121, 2.2.0+rocm5.6, 2.2.0+rocm5.7, 2.2.1, 2.2.1+cpu, 2.2.1+cpu.cxx11.abi, 2.2.1+cu118, 2.2.1+cu121, 2.2.1+rocm5.6, 2.2.1+rocm5.7, 2.2.2, 2.2.2+cpu, 2.2.2+cpu.cxx11.abi, 2.2.2+cu118, 2.2.2+cu121, 2.2.2+rocm5.6, 2.2.2+rocm5.7, 2.3.0, 2.3.0+cpu, 2.3.0+cpu.cxx11.abi, 2.3.0+cu118, 2.3.0+cu121, 2.3.0+rocm5.7, 2.3.0+rocm6.0, 2.3.1, 2.3.1+cpu, 2.3.1+cpu.cxx11.abi, 2.3.1+cu118, 2.3.1+cu121, 2.3.1+rocm5.7, 2.3.1+rocm6.0, 2.4.0, 2.4.1, 2.5.0, 2.5.1, 2.6.0, 2.7.0)\u001b[0m\u001b[31m\n",
            "\u001b[0m\u001b[31mERROR: No matching distribution found for torch==1.8.1+cu111\u001b[0m\u001b[31m\n",
            "\u001b[0m"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!python --version"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DzLjzzwbhowe",
        "outputId": "95eefeaa-bddb-4a69-d8da-2079cd9da492"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Python 3.11.12\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!python main.py --epochs 10 \\\n",
        "                --model convnext_tiny \\\n",
        "                --data_set image_folder \\\n",
        "                --data_path ../CIFAR-10-images/train \\\n",
        "                --eval_data_path ../CIFAR-10-images/test \\\n",
        "                --nb_classes 10 \\\n",
        "                --num_workers 8 \\\n",
        "                --warmup_epochs 0 \\\n",
        "                --save_ckpt true \\\n",
        "                --output_dir model_ckpt \\\n",
        "                --finetune convnext_tiny_1k_224_ema.pth \\\n",
        "                --cutmix 0 \\\n",
        "                --mixup 0 --lr 4e-4 \\\n",
        "                --enable_wandb true --wandb_ckpt true"
      ],
      "metadata": {
        "id": "_8sNl2Mb6x8_",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "43a65fc9-3069-4ec0-9d8f-82619c92360f"
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "python3: can't open file '/content/main.py': [Errno 2] No such file or directory\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!pip uninstall -y timm\n",
        "!pip install timm==0.3.2 tensorboardX six\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vdsUiZp5JHQV",
        "outputId": "aade54df-fb79-4531-8070-117fbbee7fd2"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Found existing installation: timm 0.9.12\n",
            "Uninstalling timm-0.9.12:\n",
            "  Successfully uninstalled timm-0.9.12\n",
            "Collecting timm==0.3.2\n",
            "  Using cached timm-0.3.2-py3-none-any.whl.metadata (19 kB)\n",
            "Requirement already satisfied: tensorboardX in /usr/local/lib/python3.11/dist-packages (2.6.2.2)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.11/dist-packages (1.17.0)\n",
            "Requirement already satisfied: torch>=1.0 in /usr/local/lib/python3.11/dist-packages (from timm==0.3.2) (2.2.2)\n",
            "Requirement already satisfied: torchvision in /usr/local/lib/python3.11/dist-packages (from timm==0.3.2) (0.17.2)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.11/dist-packages (from tensorboardX) (1.24.4)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.11/dist-packages (from tensorboardX) (24.2)\n",
            "Requirement already satisfied: protobuf>=3.20 in /usr/local/lib/python3.11/dist-packages (from tensorboardX) (5.29.4)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.11/dist-packages (from torch>=1.0->timm==0.3.2) (3.18.0)\n",
            "Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.11/dist-packages (from torch>=1.0->timm==0.3.2) (4.13.2)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.11/dist-packages (from torch>=1.0->timm==0.3.2) (1.13.1)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.11/dist-packages (from torch>=1.0->timm==0.3.2) (3.4.2)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.11/dist-packages (from torch>=1.0->timm==0.3.2) (3.1.6)\n",
            "Requirement already satisfied: fsspec in /usr/local/lib/python3.11/dist-packages (from torch>=1.0->timm==0.3.2) (2025.3.2)\n",
            "Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /usr/local/lib/python3.11/dist-packages (from torch>=1.0->timm==0.3.2) (12.1.105)\n",
            "Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /usr/local/lib/python3.11/dist-packages (from torch>=1.0->timm==0.3.2) (12.1.105)\n",
            "Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /usr/local/lib/python3.11/dist-packages (from torch>=1.0->timm==0.3.2) (12.1.105)\n",
            "Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /usr/local/lib/python3.11/dist-packages (from torch>=1.0->timm==0.3.2) (8.9.2.26)\n",
            "Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /usr/local/lib/python3.11/dist-packages (from torch>=1.0->timm==0.3.2) (12.1.3.1)\n",
            "Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /usr/local/lib/python3.11/dist-packages (from torch>=1.0->timm==0.3.2) (11.0.2.54)\n",
            "Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /usr/local/lib/python3.11/dist-packages (from torch>=1.0->timm==0.3.2) (10.3.2.106)\n",
            "Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /usr/local/lib/python3.11/dist-packages (from torch>=1.0->timm==0.3.2) (11.4.5.107)\n",
            "Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /usr/local/lib/python3.11/dist-packages (from torch>=1.0->timm==0.3.2) (12.1.0.106)\n",
            "Requirement already satisfied: nvidia-nccl-cu12==2.19.3 in /usr/local/lib/python3.11/dist-packages (from torch>=1.0->timm==0.3.2) (2.19.3)\n",
            "Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /usr/local/lib/python3.11/dist-packages (from torch>=1.0->timm==0.3.2) (12.1.105)\n",
            "Requirement already satisfied: triton==2.2.0 in /usr/local/lib/python3.11/dist-packages (from torch>=1.0->timm==0.3.2) (2.2.0)\n",
            "Requirement already satisfied: nvidia-nvjitlink-cu12 in /usr/local/lib/python3.11/dist-packages (from nvidia-cusolver-cu12==11.4.5.107->torch>=1.0->timm==0.3.2) (12.4.127)\n",
            "Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.11/dist-packages (from torchvision->timm==0.3.2) (11.2.1)\n",
            "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.11/dist-packages (from jinja2->torch>=1.0->timm==0.3.2) (3.0.2)\n",
            "Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.11/dist-packages (from sympy->torch>=1.0->timm==0.3.2) (1.3.0)\n",
            "Using cached timm-0.3.2-py3-none-any.whl (244 kB)\n",
            "Installing collected packages: timm\n",
            "Successfully installed timm-0.3.2\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!python /content/ConvNeXt/main.py --epochs 10 \\\n",
        "               --model convnext_tiny \\\n",
        "               --data_set cifar100 \\\n",
        "               --data_path ./data \\\n",
        "               --eval_data_path ./data \\\n",
        "               --nb_classes 100 \\\n",
        "               --num_workers 4 \\\n",
        "               --finetune convnext_tiny_1k_224_ema.pth \\\n",
        "               --lr 4e-4 \\\n",
        "               --enable_wandb true \\\n",
        "               --wandb_ckpt true\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "W0-Pr7E0I9s1",
        "outputId": "f0689ad6-0bfa-4550-aff3-c2492e59ce56"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Traceback (most recent call last):\n",
            "  File \"/content/ConvNeXt/main.py\", line 21, in <module>\n",
            "    from timm.data.mixup import Mixup\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/timm/__init__.py\", line 2, in <module>\n",
            "    from .models import create_model, list_models, is_model, list_modules, model_entrypoint, \\\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/timm/models/__init__.py\", line 1, in <module>\n",
            "    from .cspnet import *\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/timm/models/cspnet.py\", line 20, in <module>\n",
            "    from .helpers import build_model_with_cfg\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/timm/models/helpers.py\", line 17, in <module>\n",
            "    from .layers import Conv2dSame, Linear\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/timm/models/layers/__init__.py\", line 7, in <module>\n",
            "    from .cond_conv2d import CondConv2d, get_condconv_initializer\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/timm/models/layers/cond_conv2d.py\", line 16, in <module>\n",
            "    from .helpers import to_2tuple\n",
            "  File \"/usr/local/lib/python3.11/dist-packages/timm/models/layers/helpers.py\", line 6, in <module>\n",
            "    from torch._six import container_abcs\n",
            "ModuleNotFoundError: No module named 'torch._six'\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.optim as optim\n",
        "from torchvision import datasets, transforms\n",
        "from torch.utils.data import DataLoader\n",
        "import timm\n",
        "import wandb\n",
        "import matplotlib.pyplot as plt\n",
        "import random\n",
        "import numpy as np\n",
        "\n",
        "# 设置随机种子，保证结果可复现\n",
        "def set_seed(seed=42):\n",
        "    random.seed(seed)\n",
        "    np.random.seed(seed)\n",
        "    torch.manual_seed(seed)\n",
        "    torch.cuda.manual_seed_all(seed)\n",
        "\n",
        "set_seed(42)\n",
        "\n",
        "# 初始化 wandb，记录超参数\n",
        "wandb.init(project=\"convnext_cifar100\", name=\"finetune_convnext_tiny\", config={\n",
        "    \"epochs\": 10,\n",
        "    \"batch_size\": 64,\n",
        "    \"learning_rate\": 4e-4,\n",
        "    \"optimizer\": \"AdamW\",\n",
        "    \"model\": \"convnext_tiny\"\n",
        "})\n",
        "\n",
        "config = wandb.config\n",
        "\n",
        "# 图像预处理\n",
        "transform = transforms.Compose([\n",
        "    transforms.Resize((224, 224)),\n",
        "    transforms.ToTensor(),\n",
        "    transforms.Normalize((0.5071, 0.4865, 0.4409), (0.2673, 0.2564, 0.2762))\n",
        "])\n",
        "\n",
        "# CIFAR-100 数据集\n",
        "train_dataset = datasets.CIFAR100(root='./data', train=True, download=True, transform=transform)\n",
        "test_dataset = datasets.CIFAR100(root='./data', train=False, download=True, transform=transform)\n",
        "\n",
        "train_loader = DataLoader(train_dataset, batch_size=config.batch_size, shuffle=True, num_workers=2, pin_memory=True)\n",
        "test_loader = DataLoader(test_dataset, batch_size=config.batch_size, shuffle=False, num_workers=2, pin_memory=True)\n",
        "\n",
        "# 创建模型\n",
        "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
        "model = timm.create_model('convnext_tiny', pretrained=True, num_classes=100)\n",
        "model.to(device)\n",
        "\n",
        "criterion = nn.CrossEntropyLoss()\n",
        "optimizer = optim.AdamW(model.parameters(), lr=config.learning_rate)\n",
        "\n",
        "# 学习率调度器，减少学习率帮助收敛\n",
        "scheduler = optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=config.epochs)\n",
        "\n",
        "train_losses, train_accuracies = [], []\n",
        "test_accuracies = []\n",
        "\n",
        "def train(epoch):\n",
        "    model.train()\n",
        "    running_loss = 0.0\n",
        "    correct = 0\n",
        "    total = 0\n",
        "\n",
        "    for inputs, targets in train_loader:\n",
        "        inputs, targets = inputs.to(device), targets.to(device)\n",
        "        optimizer.zero_grad()\n",
        "        outputs = model(inputs)\n",
        "        loss = criterion(outputs, targets)\n",
        "        loss.backward()\n",
        "        optimizer.step()\n",
        "\n",
        "        running_loss += loss.item() * inputs.size(0)  # 乘以batch大小，计算总loss\n",
        "        _, predicted = outputs.max(1)\n",
        "        total += targets.size(0)\n",
        "        correct += predicted.eq(targets).sum().item()\n",
        "\n",
        "    avg_loss = running_loss / total\n",
        "    acc = 100. * correct / total\n",
        "    train_losses.append(avg_loss)\n",
        "    train_accuracies.append(acc)\n",
        "\n",
        "    wandb.log({\"train_loss\": avg_loss, \"train_acc\": acc, \"epoch\": epoch})\n",
        "    print(f\"[Train] Epoch {epoch}: Loss={avg_loss:.4f}, Acc={acc:.2f}%\")\n",
        "\n",
        "def test(epoch):\n",
        "    model.eval()\n",
        "    correct = 0\n",
        "    total = 0\n",
        "    with torch.no_grad():\n",
        "        for inputs, targets in test_loader:\n",
        "            inputs, targets = inputs.to(device), targets.to(device)\n",
        "            outputs = model(inputs)\n",
        "            _, predicted = outputs.max(1)\n",
        "            total += targets.size(0)\n",
        "            correct += predicted.eq(targets).sum().item()\n",
        "\n",
        "    acc = 100. * correct / total\n",
        "    test_accuracies.append(acc)\n",
        "    wandb.log({\"test_acc\": acc, \"epoch\": epoch})\n",
        "    print(f\"[Test] Epoch {epoch}: Acc={acc:.2f}%\")\n",
        "\n",
        "# 训练循环\n",
        "for epoch in range(1, config.epochs + 1):\n",
        "    train(epoch)\n",
        "    test(epoch)\n",
        "    scheduler.step()\n",
        "\n",
        "# 保存整个模型结构和参数，方便后续加载\n",
        "torch.save(model, \"convnext_cifar100_finetuned.pth\")\n",
        "wandb.save(\"convnext_cifar100_finetuned.pth\")\n",
        "\n",
        "# 绘制训练曲线\n",
        "plt.figure(figsize=(12, 5))\n",
        "plt.subplot(1, 2, 1)\n",
        "plt.plot(range(1, config.epochs + 1), train_losses, label='Train Loss')\n",
        "plt.xlabel('Epoch')\n",
        "plt.ylabel('Loss')\n",
        "plt.title('Train Loss Curve')\n",
        "plt.legend()\n",
        "\n",
        "plt.subplot(1, 2, 2)\n",
        "plt.plot(range(1, config.epochs + 1), train_accuracies, label='Train Accuracy')\n",
        "plt.plot(range(1, config.epochs + 1), test_accuracies, label='Test Accuracy')\n",
        "plt.xlabel('Epoch')\n",
        "plt.ylabel('Accuracy (%)')\n",
        "plt.title('Accuracy Curve')\n",
        "plt.legend()\n",
        "\n",
        "plt.tight_layout()\n",
        "plt.savefig(\"training_curves.png\")\n",
        "plt.show()\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 583
        },
        "id": "gzdsZvqDHfqt",
        "outputId": "17c31341-0105-48f3-8588-7ad508bf62e9"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "error",
          "ename": "ImportError",
          "evalue": "cannot import name 'OPENAI_CLIP_MEAN' from 'timm.data' (/usr/local/lib/python3.11/dist-packages/timm/data/__init__.py)",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mImportError\u001b[0m                               Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-13-a3fa6252ad0e>\u001b[0m in \u001b[0;36m<cell line: 0>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mtorchvision\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mdatasets\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtransforms\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mutils\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdata\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mDataLoader\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m \u001b[0;32mimport\u001b[0m \u001b[0mtimm\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      7\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mwandb\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      8\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mmatplotlib\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpyplot\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/timm/__init__.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;34m.\u001b[0m\u001b[0mversion\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0m__version__\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;34m.\u001b[0m\u001b[0mlayers\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mis_scriptable\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mis_exportable\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mset_scriptable\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mset_exportable\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0;34m.\u001b[0m\u001b[0mmodels\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mcreate_model\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlist_models\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlist_pretrained\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mis_model\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlist_modules\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel_entrypoint\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;31m \u001b[0m\u001b[0;31m\\\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      4\u001b[0m     \u001b[0mis_model_pretrained\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mget_pretrained_cfg\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mget_pretrained_cfg_value\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/timm/models/__init__.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;34m.\u001b[0m\u001b[0mcait\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;34m.\u001b[0m\u001b[0mcoat\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0;34m.\u001b[0m\u001b[0mconvit\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      7\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;34m.\u001b[0m\u001b[0mconvmixer\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      8\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;34m.\u001b[0m\u001b[0mconvnext\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/timm/models/convit.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     33\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;34m.\u001b[0m\u001b[0m_features_fx\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mregister_notrace_module\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     34\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;34m.\u001b[0m\u001b[0m_registry\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mregister_model\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgenerate_default_cfgs\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 35\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0;34m.\u001b[0m\u001b[0mvision_transformer_hybrid\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mHybridEmbed\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     36\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     37\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/timm/models/vision_transformer_hybrid.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     26\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;34m.\u001b[0m\u001b[0mresnet\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mresnet26d\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mresnet50d\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     27\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;34m.\u001b[0m\u001b[0mresnetv2\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mResNetV2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcreate_resnetv2_stem\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 28\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0;34m.\u001b[0m\u001b[0mvision_transformer\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0m_create_vision_transformer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mVisionTransformer\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     29\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     30\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/timm/models/vision_transformer.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     40\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mjit\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mFinal\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     41\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 42\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0mtimm\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdata\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mIMAGENET_DEFAULT_MEAN\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mIMAGENET_DEFAULT_STD\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mIMAGENET_INCEPTION_MEAN\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mIMAGENET_INCEPTION_STD\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;31m \u001b[0m\u001b[0;31m\\\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     43\u001b[0m     \u001b[0mOPENAI_CLIP_MEAN\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mOPENAI_CLIP_STD\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     44\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0mtimm\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlayers\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mPatchEmbed\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mMlp\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mDropPath\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mAttentionPoolLatent\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mRmsNorm\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mPatchDropout\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mSwiGLUPacked\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;31m \u001b[0m\u001b[0;31m\\\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mImportError\u001b[0m: cannot import name 'OPENAI_CLIP_MEAN' from 'timm.data' (/usr/local/lib/python3.11/dist-packages/timm/data/__init__.py)",
            "",
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0;32m\nNOTE: If your import is failing due to a missing package, you can\nmanually install dependencies using either !pip or !apt.\n\nTo view examples of installing some common dependencies, click the\n\"Open Examples\" button below.\n\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n"
          ],
          "errorDetails": {
            "actions": [
              {
                "action": "open_url",
                "actionText": "Open Examples",
                "url": "/notebooks/snippets/importing_libraries.ipynb"
              }
            ]
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 🏐 Conclusion\n",
        "\n",
        "* **The above setting gives a top-1 accuracy of ~95%.**\n",
        "* The ConvNeXt repository comes with modern training regimes and is easy to finetune on any dataset.\n",
        "* The finetune model achieves competitive results.\n",
        "\n",
        "* By passing two arguments you get the following:\n",
        "\n",
        "  * Repository of all your experiments (train and test metrics) as a [W&B Project](https://docs.wandb.ai/ref/app/pages/project-page). You can easily compare experiments to find the best performing model.\n",
        "  * Hyperparameters (Configs) used to train individual models.\n",
        "  * System (CPU/GPU/Disk) metrics.\n",
        "  * Model checkpoints saved as W&B Artifacts. They are versioned and easy to share.\n",
        "\n",
        "  Check out the associated [W&B run page](https://wandb.ai/ayut/convnext/runs/16vi9e31). $→$"
      ],
      "metadata": {
        "id": "350MmZgtBVWy"
      }
    }
  ]
}